Method and apparatus for adaptive edge-based scan line interpolation using 1-D pixel array motion detection

ABSTRACT

The present invention generally relates to the conversion of a picture data in an interlaced format to a progressive format, and more particularly, to an adaptive edge-based scan line interpolation method and apparatus for improving the picture quality in a display system. The present invention provides a method and apparatus for converting interlaced scanning images to progressive images by using simple motion detection procedure and adaptive edge-interpolation. Therefore, the present invention is capable of performing the interlaced-to-progressive conversion with less hardware and buffer cost. Besides, the present invention performs the motion detection by examining the 1-D (one-dimension) pixel array at the adjacent fields. Therefore, the computation of the interlaced-to-progressive conversion of the present invention is less complex than the prior art. Furthermore, the video image of the present invention has a sharper video image with edge preservation and flicker reduction.

FIELD OF THE INVENTION

The present invention generally relates to the conversion of a picturedata in an interlaced format to a progressive format, and moreparticularly, to an adaptive edge-based scan line interpolation methodand apparatus for improving the picture quality in a display system.

BACKGROUND OF THE INVENTION

As well known, NTSC color television systems adopted an interlacedscanning system to reduce the transmission bandwidth requirements. Theinterlaced scanning video displays half of each frame (called a field)every 1/60 second, followed by the other field 1/60 second later.Referring to FIG. 1, it illustrates the interlaced scanning video formatwith a top field and a bottom field of the prior art. The top fieldcontains all the odd-numbered horizontal lines, and the bottom fieldcontains all the even-numbered lines. However, the interlaced scanningmanner has drawbacks of visual artifacts, including edge flicker,shimmering and diagonal jaggedness. Deinterlacing can overcome aboveproblems to improve the appearance of the interlaced scanning video byconverting the interlaced scanning video format into a progressivescanning format. Besides, many applications, such as PC displays,projection and high-definition television employ progressive-scantechnology. To convert the interlaced scanning pictures to progressivescanning pictures, the missing lines of each field (for example, theeven-numbered lines in the top field or the odd numbered lines in thebottom field) are generated from the interlaced scanning video signals.

One method for the interlaced-to-progressive conversion is a statictechnique that uses the same overall conversion regardless of the imagesources or contents. This method uses the techniques of linereplication, vertical filtering and field merging. Line replicationrepeats each horizontal line in a field to create a complete frame.Vertical filtering creates missing lines by filtering a number of nearbylines. Field merging takes lines from the previous field and insertsthem into the current field to construct the frame. However, theseapproaches result in visual artifacts such as serrated edges for motionobjects in the video frame.

To overcome these drawbacks, adaptive deinterlacing techniques aredeveloped. Such adaptive deinterlacing techniques use motion analysis toselect the optimum method for different parts of the image. Adaptivedeinterlacing techniques first check both the current and one or moreprevious fields to determine the motion contents. In image sequenceswith little or no motion, missing lines take pixels values from theprevious field. If there is a significant motion, an edge adaptivespatial interpolation method is used to construct the missing lines.

However, the above-mentioned approaches using adaptive interlacingtechniques have complex computation procedure to determine the motioncontents and interpolate the missing pixels values. U.S. Pat. No.5,019,903 describes a technique which only considers how to interpolatethe missing pixels without details to the motion detection circuit. U.S.Pat. No. 5,473,383 uses extra buffers to store the signalsrepresentative of the motion signals. U.S. Pat. No. 5,592,231 uses amedian filter to select motion signal and applies five-edgeinterpolations. U.S. Pat. No. 5,532,751 uses two fields in the sameframe to detect motion. U.S. Pat. No. 5,339,109 uses three-edgeinterpolation without order but does not consider the sharpness of theedges. However, the improper edge interpolation will blur the image.

SUMMARY OF THE INVENTION

To overcome the aforementioned problems, the present invention providesa method for converting interlaced scanning images to progressive imagesby using simple motion detection procedure and adaptiveedge-interpolation. Therefore, it is an object of the present inventionto implement the interlaced-to-progressive conversion with less hardwareand buffer cost. The present invention performs the motion detection byexamining the 1-D (one-dimension) pixel array at the adjacent fields. Ifthe missing pixel is found to be steady portion of the image, its valueis replaced by the corresponding value at the field immediatelypreceded. Otherwise, if the image has motion in the part of the missingpixels, the missing pixel value is interpolated by the adaptiveedge-interpolation. The adaptive edge-interpolation interpolates themissing pixel along the edge direction that is the minimum of the threeedges crossing the missing pixel. With respect to the videos with MPEG-2(Moving Picture Expert Groups-2) format, the present invention alsoprovides a memory organization for interlace-to-progressive conversionto minimize the buffer requirement. Therefore, the present invention hasa sharper video image with edge preservation and flicker reduction andhas simple computation and low buffer cost since only 1-D pixel array isexamined to determine the motion signal.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and theadvantages thereof, reference is now made to the following descriptionstaken in conjunction with the accompanying drawings which illustrate oneor more embodiments of the present invention, wherein

FIG. 1 illustrates the format of the interlaced scanning video image ofthe prior art;

FIG. 2 illustrates the block diagram of the preferred embodiment of theinterlace-to-progressive conversion of the present invention;

FIG. 3 illustrates the block diagram of the adaptive edge-basedinterlace-to--progressive conversion of the present invention;

FIG. 4 illustrates a graphical example of the process to determine amotion signal of the present invention;

FIG. 5 illustrates the block diagram of the flow chart of the adaptiveedge interpolation of the present invention;

FIG. 6 illustrates a graphical example of a MPEG (Moving Picture ExpertGroups) video sequence for interlace-to-progressive conversion of thepresent invention;

FIG. 7 illustrates the block diagram of the preferred embodiment of thememory organization of the present invention for the MPEG video format;and

FIG. 8 illustrates a graphical example of the memory content updatingprocess for the MPEG video format.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 2, it illustrates the block diagram of the preferredembodiment of the interlace-to-progressive conversion of the presentinvention. The video signal format from either the video capture (22) orthe decoded MPEG video (23) is an interlaced format. To simultaneouslydisplay the interlaced video signal format (22 or 23) and progressiveVGA graphic signal (21) on a progressive monitor like VGA monitor (26),the interlaced signals (22 or 23) are converted to the progressiveformat by an interlace-to-progressive conversion (24). The convertedvideo signal is then mixed with the graphic signal (21) by a mixer (25)and then displays on a VGA monitor (26).

Referring to FIG. 3, it illustrates the block diagram of the adaptiveedge-based interlace-to-progressive conversion of the present invention.For each pixel in the lines to be interpolated in field i, where idenote a field of a first polarity (32), motion detection is firstemployed to determine whether the pixel is in the motion region of theimage by comparing the |a-b| with the predetermined threshold (37) by acomparator (36). The comparing result is determined by a switch (35)whether a field merging or an edge-based interpolation is used togenerate the pixel value.

The motion detection mechanism uses the two consecutive fields i-1 andi+1 that immediately precede and follow field i, where i-1 and i+1denotes fields of a second polarity (33). That is to say, when icorresponds to the bottom field, fields i-1 and i+1 are consecutive topfields; and when i corresponds to the top field, fields i-1 and i+1 areconsecutive bottom fields.

Referring to FIG. 4, it illustrates an non-limited graphical example todetermine the motion signal for the missing pixel x2 of the presentinvention. The motion signal is detected by examining the contents of an1-D pixel array (three pixels in this example) from field i-1 thatimmediately precedes and field i+1 that immediately follows field i. Forexample, with respect to the missing pixel x2, the correspondingthree-pixel array in field i-1 consists of pixels b1, b2 and b3 and thecorresponding three-pixel array in field i+1 consists of pixels j1, j2and j3. A sum of absolute difference between the two pixel arrays isgenerated according to the equation as follows:

    det=|j1-b1|+|j2-b2|+|j3-b3.vertline.                                                     (1)

This sum signal det is compared with a threshold value. If the sumsignal is smaller than the threshold value, then the pixel x2 is assumedto be in the steady portion of the image. The pixel value of x2 isreplaced by the value of corresponding pixel b2 in the field i-1. If thesum signal exceeds than the threshold value, then it is concluded thatthere is an effective relative image motion in this part of the image.The pixel value of x2 is estimated by edge-interpolation.

To reduce the computation cost, the edge-interpolation method of thepresent invention considers three edges that cross the missing pixel.For example, with respect to the missing pixel x2, the three edges arethe vertical edge e2-f2 and diagonal edges e1-f3 and e3-f1. Referring toFIG. 5, it illustrates the flow chart of the adaptive edgeinterpolation. The edge values of the diagonal edges e1-f3 and e3-f1 arefirst computed (51), then the edge value of the vertical edge e2-f2 iscomputed (52). The edge direction is selected to be the minimum of thethree absolute values of the difference, i.e. |e1-f3|, |e2-f2| and|e3-f1|. If two or more edges have the same minimum absolute value ofthe difference, the edge direction is selected to be the edge that waslast computed and has minimum value (53).

After the edge direction is found, the missing pixel is determined bycomparing the edge value with a threshold value (54). If the edge valueis greater than a threshold value (55), the value of the missing pixelx2 is one of the two pixel values in the edge direction and a shaperedge exists. Therefore, no interpolation is performed to ensure thesharper edge is not blurred due to interpolation. The value of missingpixel x2 is interpolated to be the equally weighted sum of the two pixelvalues (the average of the two pixel values) in the edge direction ifthe edge value is less than a threshold value (56). For example, if theedge direction is e2-f2 and its edge value is less than the thresholdvalue, the value of the missing pixel x2 is equal to (e2+f2)/2. If theedge value in the edge direction is greater than the threshold value,the value of the missing pixel x2 is equal to either one of the twopixels in the edge direction. For example, if the edge direction ise2-f2 and its edge value is greater than the threshold value, the valueof the missing pixel x2 is equal to the value of pixel e2 or f2.

For a video signal with MPEG (Moving Picture Expert Groups) format, thebuffers required by the interlace-to-progressive conversion are sharedwith MPEG decoder since the interlace-to-progressive conversion occursafter the MPEG decoding process as illustrated in FIG. 2. Referring toFIG. 6, it illustrates a graphical example of a MPEG video sequence forthe interlace-to-progressive conversion of the present invention. EachI-, B- and P- picture consists of top field and bottom field. As shownin FIG. 6, the display order is similar to the TV sequence. When onlytwo consecutive B-pictures exist between I- and P-pictures, which occursin most of the MPEG video stream, four memory blocks are used to meetthe buffer requirements. Referring to FIG. 7, it illustrates thepreferred embodiment of the memory organization of the present inventionfor the MPEG video format. Four memory blocks are used for the memoryorganization. Two blocks are for the interlace-to-progressive conversionand two blocks are for storing the MPEG picture data to be decoded

Referring to FIG. 8, it illustrates a graphical example of memorycontent updating process for FIG. 6 of the present invention. MEM1 (71)and MEM4 (74) blocks are for I-picture and P-picture, and MEM2 (72) andMEM3 (73) blocks are for B-pictures, respectively. New I-, P- andB-pictures are stored in the corresponding blocks for that type ofpicture in their display order. As illustrated in FIG. 8, when a newframe picture is received, the old picture in the corresponding memoryblock is replaced by the new picture. For example, when P6 is received,the data of IO is replaced by P6.

As illustrated in FIG. 8, the three consecutive fields used in theinterlace-to-progressive conversion process for MPEG video format arefrom two consecutive pictures in the display order if the consecutiveB-picture numbers are not greater than two. For example, to interpolatetop field of B2, B1 and B2 are used as reference pictures. If theconsecutive B-picture numbers are greater than two, one of the memoryblocks that stores B-picture is used to store the newly decoded data.Therefore, the interpolation of bottom field of B-pictures in MEM2 (72)uses top field in MEM4 (74) if MEM3 (73) are used for storing decodedpicture data, and the interpolation of top field of B-pictures in MEM3(73) uses bottom field in MEM1 (71) if MEM2 (72) are used for storingdecoded picture data.

The present invention provides a method and mechanism for convertinginterlaced scanning images to progressive images by using simple motiondetection procedure and adaptive edge-interpolation. Therefore, thepresent invention is capable of performing the interlaced-to-progressiveconversion with less hardware and buffer cost. Besides, the presentinvention performs the motion detection by examining the 1-D(one-dimension) pixel array at the adjacent fields. Therefore, thecomputation of the present invention is less complex than the prior art.Furthermore, the video image of the present invention has a sharpervideo image with edge preservation and flicker reduction.

Although the present invention and its advantage have been described indetail, it should be understood that various changes, substitutions andalternations can be made herein without departing from the spirit andscope of the invention as defined by the appended claims.

What is claimed is:
 1. A method for generating a progressive scanningvideo image, comprising the steps of:(a) selecting a first 1-D pixelarray in a first image field and a second 1-D pixel array in a secondimage field; (b) computing a value between said first and second arrayby comparing the sum of the absolute value of the difference betweensaid first and second array with a predetermined threshold value; and(c) determining an interpolated pixel value according to said valueobtained in step (b) and interpolating said interpolated pixel valueinto an interlaced video image to be converted to the progressivescanning video image.
 2. The method as claimed in claim 1, wherein thecontents of said 1-D pixel arrays are the pixels at the correspondingimage lines of interpolated pixel.
 3. The method as claimed in claim 1,wherein no motion between said first and second array has occurred ifsaid sum of the absolute value of the difference between said first andsecond array is less than said predetermined threshold value.
 4. Themethod as claimed in claim 1, wherein a motion between said first andsecond array has occurred if the said sum of the absolute value of thedifference between said first and second array is greater than saidpredetermined threshold value.
 5. The method as claimed in claim 1,wherein said interpolated pixel value is replaced by the value of thecorresponding pixel at said first image field if there is no motionbetween said first and second array.
 6. The method as claimed in claim1, wherein said interpolated pixel value is predicted by edge-basedinterpolation if there is a motion between said first and second array.7. The method as claimed in claim 6, wherein said prediction of saidinterpolated pixel value further comprises the steps of:(d) calculatingvalues of |x(k)-y(-k)|, wherein x(k) is the value of the pixel at theimage line above the interpolated pixel at the same field, and k isequal to -N, . . . -2, -1, 1, 2 . . . , N, and y(k) is the value of thepixel at the image line below the interpolated pixel at the same field;(e) calculating the value of |x(0)-y(0)|; (f) determining an edgedirection by selecting the minimum value of said values of |x(k)-y(-k)|,wherein k=-N, . . . , -1, 0, 1, . . . N; and (g) determining the valueof a missing pixel by comparing said values obtained from steps of (d)and (e) with a predetermined threshold value.
 8. The method as claimedin claim 7, wherein the value of said missing pixel is one of said pixelvalues in said edge direction if the value of said edge direction isgreater than said predetermined threshold value.
 9. The method asclaimed in claim 7, wherein the value of said missing pixel is anaverage of said pixel values in said edge direction if the value of saidedge direction is less than said predetermined threshold value.
 10. Themethod as claimed in claim 7, wherein the pixel pair that was lastlycomputed is chosen to predict said missing pixel value if two or morevalues of |x(k)-y(-k)| are the same.
 11. An apparatus for generating aprogressive scanning video image, comprising:a first 1-D pixel array ina first image field for storing pixels at the corresponding image linesof the pixel to be interpolated; a second 1-D pixel array in a secondimage field for storing pixels at the corresponding image lines of thepixel to be interpolated; a comparator connected to said 1-D pixelarrays for comparing the sum of the absolute value of the differencebetween said first and second array with a predetermined threshold valueand generating a value; and an interpolated means for interpolating aninterpolated pixel value according to said value.
 12. The apparatus asclaimed in claim 11, wherein no motion between said first and secondarray has occurred if said sum of the absolute value of the differencebetween said first and second array is less than said predeterminedthreshold value.
 13. The apparatus as claimed in claim 11, wherein amotion between said first and second array has occurred if the said sumof the absolute value of the difference between said first and secondarray is greater than said predetermined threshold value.
 14. Theapparatus as claimed in claim 11, wherein said interpolated pixel valueis replaced by the value of the corresponding pixel at said first imagefield if there is no motion between said first and second array.
 15. Theapparatus as claimed in claim 11, wherein said interpolated pixel valueis predicted by edge-based interpolation if there is a motion betweensaid first and second array.
 16. The apparatus as claimed in claim 15,further comprising:a means for calculating values of |x(k)-y(-k)|,wherein x(k) is the value of the pixel at the image line above theinterpolated pixel at the same field, and k is equal to -N, . . . -2,-1, 1, 2 . . . , N, and y(k) is the value of the pixel at the image linebelow the interpolated pixel at the same field; a means for calculatingthe value of |x(0)-y(0)|; a means for determining an edge direction byselecting the minimum value of said values of |x(k)-y(-k)|, whereink=-N, . . . , -1, 0, 1, . . . N; and a comparison means for determiningthe value of a missing pixel by comparing said values with apredetermined threshold value.
 17. The apparatus as claimed in claim 16,wherein the value of said missing pixel is one of said pixel values insaid edge direction if the value of said edge direction is greater thansaid predetermined threshold value.
 18. The apparatus as claimed inclaim 16, wherein the value of said missing pixel is an average of saidpixel values in said edge direction if the value of said edge directionis less than said predetermined threshold value.
 19. The apparatus asclaimed in claim 16, wherein the pixel pair that was lastly computed ischosen to predict said missing pixel value if two or more values of|x(k)-y(-k)| are the same.